In the last few years there has been a renewed interest in 3D contents in the general public. Reasons for this development include the aggressive marketing of 3D movies and TVs as well as the availability of low cost consumer stereoscopic and depth cameras. From a research standpoint, the additional images or direct depth measurement from novel sensors provide great opportunities for many vision tasks. This is especially true for those tasks that traditionally relied on 2D information often captured from a single viewpoint. The goal of this special issue was therefore to attract the latest research in 3D imaging, processing and modelling. In response to the call for contributions we received a total of 60 submissions, out of which the following 16 papers were selected during the reviewing process.

Sensors and techniques for the acquisition of 3D information through optical means continue to evolve: the first four papers in this issue introduce innovations and improvements in this area, as well as a framework for sensor assessment. The paper “Depth from Refraction Using a Transparent Medium with Unknown Pose and Refractive Index” by Chen et al. describes a method to recover the scene depth using a small number of images with and one without a slab of transparent medium between the camera and the target object. Lu et al. propose “A 3D Imaging Framework Based on High-Resolution Photometric-Stereo and Low-Resolution Depth” which reconstructs high quality normals and depth using a combined method. Global illumination effects in 3D scanning are studied in “A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus” by Gupta et al. The authors investigate the errors caused by global illumination and present a robust structured-light based 3D scanning method. The paper “GD&T-based Characterization of Short-range Non-contact 3D Imaging Systems” by MacKinnon et al. proposes test metrics, artifacts, and procedures for quantifying and comparing performance of optical rangefinders. The framework’s application and usefulness is demonstrated on actual systems.

This issue also includes two studies on the problem of 3D registration. The paper “A Theory of Minimal 3D Point to 3D Plane Registration and its Generalization” by Ramalingam et al. discusses a family of minimal solutions for 3D-to-3D registration problems. The systematic investigation on the 3D data registration benefits diverse 3D applications such as registration of parametric models with 3D point clouds and registration of depth maps obtained from depth sensors. In “Fully Automatic Registration of Image Sets on Approximate Geometry” by Corsini et al., the problem of automatically registering a set of images against a 3D scan or model of an environment is discussed. The paper presents details of a complete pipeline for the registration that benefits both researchers and practitioners.

The analysis and interpretation of 3D data remains an important area for research, and is represented here by six papers. In “Online Detection of Repeated Structures in Point Clouds of Urban Scenes for Compression and Registration”, Friedman et al. use domain-specific constraints found in laser scans of urban areas, and perform on-line detection of repetitive architectural features such as windows. The identification of such patterns leads to improved compression and registration methods. Rodolà et al. propose in “A Scale Independent Selection Process for 3D Object Recognition in Cluttered Scenes” a 3D recognition pipeline that computes and filters distinctive local descriptors, combined with game-theoretical methods. The automatic detection of local points of interest that are candidates for establishing correspondences is the focus of “A Machine-learning Approach to Keypoint Detection and Landmarking on 3D Meshes” by Creusot et al. Machine learning techniques are used to learn landmark detection functions, and results are shown for 3D face datasets. Yu et al.’s paper “A Performance Evaluation of Volumetric 3D Interest Point Detectors” investigates the performance of popular interest point detectors for 3D volumetric data. A new score combining both positional accuracy and repeatability is proposed for comparison. In “Performance Evaluation of 3D Keypoint Detectors” by Tombari et al. the three fundamental aspects of repeatability, distinctiveness, and computational efficiency are evaluated for several 3D keypoint detectors. The paper “Feature-preserved 3D Canonical Form” by Lian et al. proposes a method for generating a feature-preserved 3D canonical form of a non-rigid object for 3D shape retrieval. It relies on segmenting the input model into pieces which are rearranged rigidly according to a standard canonical form, and stitched together using a technique to diffuse errors in the deformation-gradient domain.

The processing and analysis of sequences of 3D images of non-rigid objects is an important emerging trend in the field: four papers illustrate current topics of interest. The method proposed in “Direct Model-based Tracking of 3D Object Deformations in Depth and Color Video” by Jordt et al. takes advantage of combined depth and video data flows for tracking objects by applying a NURBS deformation model to the input image, and not just on feature points. Results show the ability to handle occlusions by the object itself or by its environment. In their paper “Global Non-rigid Alignment of Surface Sequences”, Budd et al. use a shape similarity tree to align non-rigidly deformed geometry in non-sequential frames and multiple sequences. In “Consistent Binocular Depth and Scene Flow with Chained Temporal Profiles,” Hung et al. show that long-range temporal constraints significantly improve the depth and scene flow consistency, and produce reliable depth and motion estimation from binocular videos. Finally, Yan et al.’s “Depth Mapping for Stereoscopic Videos” applies 3D image analysis techniques in order to map the depth range of a stereoscopic video from its original intended viewing configuration to different ones resulting for example from smaller screens or a closer viewing distance.

Finally, we would like to thank everybody who contributed to this issue. We thank in particular all authors, reviewers, as well as the editors-in-chief and the editorial staff. We are also very grateful to IJCV for hosting this special issue. We are confident that it provides the reader with a representative portrait of current research issues and approaches, and will stimulate further work in 3D imaging, processing and modelling in the computer vision community.