DeepStereoBrush: Interactive Depth Map Creation
Citation:
Knorr, Sebastian; Hudon, Matis; Cabrera, Julian; Sikora, Thomas; Smolic, Aljosa, DeepStereoBrush: Interactive Depth Map Creation, International Conference on 3D Immersion, 2018, (Received the Lumiere Award for the best scientific paper)., Belgium, 2018Download Item:
DeepStereoBrush_Stereopsia-6.pdf (PDF) 7.531Mb
Abstract:
In this paper, we introduce a novel interactive depth map creation approach for image sequences which uses depth-scribbles
as input at user-defined keyframes. These scribbled depth values are then propagated within these keyframes and across
the entire sequence using a 3-dimensional geodesic distance
transform (3D-GDT). In order to further improve the depth
estimation of the intermediate frames, we make use of a convolutional neural network (CNN) in an unconventional manner. Our process is based on online learning which allows us
to specifically train a disposable network for each sequence
individually using the user generated depth at keyframes along
with corresponding RGB images as training pairs. Thus, we
actually take advantage of one of the most common issues in
deep learning: over-fitting. Furthermore, we integrated this
approach into a professional interactive depth map creation
application and compared our results against the state of the
art in interactive depth map creation.
URI:
https://v-sense.scss.tcd.ie/research/vfx-animation/deepstereobrush-interactive-depth-map-creation/http://hdl.handle.net/2262/90494
Sponsor
Grant Number
Science Foundation Ireland (SFI)
15/RP/2776
Author's Homepage:
http://people.tcd.ie/smolicaDescription:
PUBLISHEDBelgium
Author: Smolic, Aljosa
Other Titles:
International Conference on 3D Immersion, 2018, (Received the Lumiere Award for the best scientific paper).Type of material:
Conference PaperURI:
https://v-sense.scss.tcd.ie/research/vfx-animation/deepstereobrush-interactive-depth-map-creation/http://hdl.handle.net/2262/90494
Collections:
Availability:
Full text availableKeywords:
Depth estimation, 2D-to-3D conversion, Deep learning, CNN, Interactive depth map creation, Geodesic distance transformSubject (TCD):
Creative Technologies , Multimedia & CreativityLicences: