High Resolution Light Field Recovery with Fourier Disparity Layer Completion, Demosaicing, and Super-Resolution

Abstract

In this paper, we present a novel approach for recovering high resolution light fields from input data with many types of degradation and challenges typically found in lenslet based plenoptic cameras. Those include the low spatial resolution, but also the irregular spatio-angular sampling and color sampling, the depth-dependent blur, and even axial chromatic aberrations. Our approach, based on the recent Fourier Disparity Layer representation of the light field, allows the construction of high resolution layers directly from the low resolution input views. High resolution light field views are then simply reconstructed by shifting and summing the layers. We show that when the spatial sampling is regular, the layer construction can be decomposed into linear optimization problems formulated in the Fourier domain for small groups of frequency components. We additionally propose a new preconditioning approach ensuring spatial consistency, and a color regularization term to simultaneously perform color demosaicing. For the general case of light field completion from an irregular sampling, we define a simple iterative version of the algorithm. Both approaches are then combined for an efficient super-resolution of the irregularly sampled data of plenoptic cameras. Finally, the Fourier Disparity Layer model naturally extends to take into account a depth-dependent blur and axial chromatic aberrations without requiring an estimation of depth or disparity maps.