This special issue contains selected papers from the IEEE International Conference on Computer Vision (ICCV), which was held in Santiago de Chile in December of 2015, and was jointly sponsored by the Institute of Electrical and Electronics Engineers (IEEE) and the Computer Vision Foundation (CVF). In 2015, there were 1700 submissions to ICCV, from which 526 were selected for oral or poster presentation by a team of 115 area chairs and 1300+ reviewers. From the accepted papers, area chairs recommended a subset that were especially appreciated by the reviewers for their technical contributions, and for their expected impacts to the field. We, program chairs from that conference and guest editors of this special issue of IJCV, invited authors of those papers to submit a longer manuscript to this special issue. The submitted papers underwent a second review process to guarantee that all papers included in this special issue be rigorously peer-reviewed. As a result, 9 papers were accepted for publication. These papers cover areas like low-level vision and image processing; 3D computer vision; computational photography; and visual recognition. We found these papers to be delightful, and we hope that you will, too.

The first paper, “Holistically-Nested Edge Detection. doi:10.1007/s11263-017-1004-z,” by Saining Xie and Zhuowen Tu, addresses the problem of detecting edges and object boundaries in natural images. Specifically, the authors propose a new edge detection algorithm based on a fully convolutional neural model, which addresses two important issues: (1) holistic image training and prediction; and (2) multi-scale and multi-level feature learning. This paper received the best paper award honorable mention at the ICCV 2015 conference.

The second paper, “Mutual-Structure for Joint Filtering. doi:10.1007/s11263-017-1021-y,” by Xiaoyong Shen, Chao Zhou, Li Xu and Jiaya Jia, addresses the problem of filtering a target image by using structural information from a reference image. Specifically, the authors propose to use mutual structural information contained in both images to address the challenge that edge information may be different in both images.

The third paper, “Depth Sensing using Geometrically Constrained Polarization Normals. doi:10.1007/s11263-017-1025-7,” by Achuta Kadambi, Vage Taamazyan, Boxin Shi and Ramesh Raskar, addresses the problem of recovering 3D geometry from polarimetric properties of reflected light. In particular, the authors use prior geometric information from coarse depths maps to constrain shape information extracted from polarization cues.

The fourth paper, “3D Time-Lapse Reconstruction from Internet Photos. doi:10.1007/s11263-017-1003-0,” by Ricardo Martin-Brualla, David Gallup and Steven Seitz, addresses the problem of building a 3D time-lapse video sequence from a photo collection of a landmark. The key contribution of this work is to create 3D time-lapse video where the virtual camera moves continuously in time and space.

The fifth paper, “Automatic Registration of Images to Untextured Geometry using Average Shading Gradients. doi:10.1007/s11263-017-1022-x,” by Tobias Plötz and Stefan Roth, addresses the problem of registering photographs to 3D models in the challenging case where no texture information is available. For this purpose, the authors propose a new technique to estimate the average gradient magnitude and use it within a sparse feature matching framework for image-to-geometry registration.

The sixth paper, “Global, Dense Multiscale Reconstruction for a Billion Points. doi:10.1007/s11263-017-1017-7,” by Benjamin Ummenhofer and Thomas Brox, addresses the problem of reconstructing a 3D surface from a set of oriented points with scale information. In particular, the authors propose a variational approach for surface reconstruction specifically designed for scenarios with nonuniform point densities due to images taken from different distances.

The seventh paper, “How Good Is My Test Data? Introducing Safety Analysis for Computer Vision. doi:10.1007/s11263-017-1020-z” by Oliver Zendel, Markus Murschitz, Martin Humenberger and Wolfgang Herzner, studies the problem of assessing the quality of test data. Specifically, the authors adapt a standard procedure devised by the safety community to identify system failure or performance loss. The result is a checklist with more than 900 hazards that can be used to analyze existing datasets. The framework is applied to stereo datasets.

The eighth paper, “Ask Your Neurons: A Deep Learning Approach to Visual Question Answering. doi:10.1007/s11263-017-1038-2,” by Mateusz Malinowski, Marcus Rohrbach and Mario Fritz, proposes a deep learning approach to the problem of answering questions about real-world images. Specifically, the authors propose a scalable, jointly trained, end-to-end formulation that combines image representations with natural language processing.

The last paper, “Learning Image Representations Tied to Egomotion from Unlabeled Video. doi:10.1007/s11263-017-1001-2,” by Dinesh Jayaraman and Kristen Grauman, studies how images of objects and scenes behave in response to egomotions. Specifically, the authors propose a new visual learning paradigm that exploits proprioceptive motor signals to train visual representations from egocentric video with no manual supervision.

The guest editors would like to thank all the authors for their valuable contributions to this special section. We also thank the reviewers of these manuscripts, who made many helpful suggestions for changes to the papers. We also thank the editorial staff of IJCV for their support. We hope that the papers selected in this special section will become useful resources for researchers and practitioners in computer vision, pattern recognition, and beyond.