Simultaneously retargeting and super-resolution for stereoscopic video

Abstract

This paper presents a novel approach that is able to resize stereoscopic video to fit various display environments with different aspect-ratios, while preserving the prominent content, keeping temporally consistent, adapting depth, as well as increasing the resolution. Our proposed approach can deal with retargeting and super-resolution problems simultaneously via replacing the down-sampling matrix appearing in super-resolution algorithm with a novel one, named as content-aware-sampling matrix, derived from retargeting method. The new matrix can sample the image into any resolution while preserving its important information as much as possible. Our approach can be roughly subdivided into three steps. In the first step, we calculate the overall saliency map for a shot, while considering the conspicuous information such as motion, depth, and structures. In the second step, given a certain resolution, we compute the retargeting parameters by a global optimization and formulate them into a matrix. Finally, we substitute the matrix into the objective function of super-resolution to achieve high visual quality images with expected resolution. In addition, we propose a novel single image super-resolution method inspired by a blind image deblurring method. The experimental results based on user studies verify the effectiveness of our approach. And the comparisons with the-state-of-the-art single image super-resolution methods validate the potential of our super-resolution method.