Exploration in Light Field processing and editing

Abstract

Light fields have been used in computer science research for the better part of the last three decades, and the range of applications available is ever-growing. There exist several capture methods producing output images with varying characteristics in terms of resolution or baseline, but the usability of these setups is negatively correlated with the overall quality of the output. One such method is plenoptic cameras, easy to use and without a need for complex calibrations, and resulting in images with low resolution, small baseline, and a number of visual artefacts which are still partially unaddressed. Some software solutions exist to counteract these hardware issues, but they are limited, and it has had an impact on the majority of light field research applications.

In this thesis, we take a closer look at these types of light fields in three ways, to study ways to enhance their visual quality, to use them for the purpose of colour editing, and to compare them to more modern light field methods. First we analyse the images captured by a Lytro Illum camera and the visual artefacts affecting them. Based on this we propose a set of tools to extract those images from RAW camera data, and perform demultiplexing, white balance, colour correction and denoising on them. Second, we use these enhanced images and perform some colour editing using a method called soft colour segmentation. Third we study the possibilities of NeRF, a new method to generate light fields, and compare it with previous traditional light field methods for view synthesis and depth estimation, to showcase the benefits it could bring for easier high quality light field capture.