Hybrid Pipeline Infinity Laplacian Plus Convolutional Stage Applied to Depth Completion

Abstract

Interpolation of the sparse depth map is a fundamental task for applications such as video games or autonomous vehicles. Frequently, depth maps present a lack of information due to sensor misinterpretations or occlusion. Traditional image processing methods have been used to tackle this interpolation problem. Nowadays, convolutional neural networks have been used to solve this problem with diverse approaches. On one hand, traditional methods are simple to implement, but the model is constructed by the designer. On the other hand, convolutional networks need a large database in order to train a model and are far to be implemented in a real platform such as FPGA. The question is if it is possible to state a hybrid model that considers the advantages of both points of view. Our proposal considers the infinity Laplacian (or AMLE) which is the most straightforward operator that satisfies a set of simple axioms. Embedding a reference color image with an anisotropic metric, we propose a three-stage pipeline to interpolate depth maps: color features selection, AMLE solution, and post-filtering. Inspired by convolutional networks, we constructed two convolution max pooling stages for color features selection. Taking those features, we used them as the color reference image, which defines the metric $g_{ij}$, and then we solved the AMLE. Finally, a second convolutional stage was applied to the interpolated map. The obtained results show that the proposed convolutional stage can be replaced by a median filter. This fact increases the performance of the complete pipeline, and this pipeline outperforms contemporary methods in the KITTI dataset showing clearly the contribution of a convolutional preprocessing stage. The convolution stage enforces strong edges and smooth textures in the reference image to let the anisotropic diffusion process fill in the empty regions bounded by strong edges.