Statistical framework for multi sensor fusion and 3D reconstruction

Abstract

Multi-view 3D reconstruction is an area of computer vision where multiple images are taken of an object and information in those images is used to generate a 3D model describing the shape and size of that object. The ability to automatically generate 3D models of objects has many uses from content creation for games to object recognition and is a first step in many other computer vision tasks like markerless motion capture. In this thesis a new framework to achieve 3D reconstruction is presented. This framework is based on the generalised Radon transform and is linked to kernel density estimation. A new smooth differentiable function is defined that can be optimised using gradient ascent algorithms. The framework is applied to two applications; firstly to computing the visual hull, a 3D reconstruction from multiple silhouettes and secondly, to generate a 3D reconstruction from depth information. The framework is capable of overcoming the considerable noise present in depth data to generate an accurate 3D reconstruction. The framework is extended to optimise camera alignment parameters in a multicamera system. Existing techniques for calculating camera parameters can be prone to error. This extension optimises these initial estimates of the camera parameters to facilitate accurate 3D reconstructions in real environments. Finally two data-sets were generated and captured to test and evaluate all the algorithms developed.