Learning Depth from Light Field via Deep Convolutional Neural Network

Abstract

Depth estimation based on light field imaging is a new way of depth estimation. The abundant data of light field paves the way for deep learning methods to play a role. In recent years, deep convolutional neural network (CNN) has shown great advantages in extracting image texture features. Inspired by this, we design a deep CNN with EPI synthetic images as input for depth estimation, which is called EENet. Under the constrains of epipolar geometry, pixels corresponding to a common object point are distributed in a straight line in the epipolar plane image (EPI), and the EPI synthetic image is easier to extract features by convolution kernel. Our EENet has the structure characteristics of multi-stream inputs and skip connections. Specifically, the horizontal EPI synthetic image, the vertical EPI synthetic image and the central view image are first generated from the light field, and input into the three streams of EENet respectively. Next, the U-shaped neural network is designed to predict the depth information, that is, the convolution and pooling blocks are used to encode the features, while the deconvolution layer and convolution layer are combined to decode features and recover the depth information. Furthermore, we employ skip connections between the encoding layers and the decoding layers to fuse the shallow location features and deep semantic features. Our EENet is trained and tested on the light field benchmark, and has obtained good experimental results.