User-assisted reflection detection and feature point tracking

Abstract

Reflections in image sequences violate the single layer model used by most current image processing techniques. As a result reflec- tions cause many techniques to fail e.g. detection, tracking, motion estimation, etc. Recent work was proposed by Ahmed et al. [5] to detect reflections. Their technique is robust to pathological motion and motion blur. This paper has three main contributions. The first simplifies and fully automates the technique of Ahmed et al. User feedback is common in post-production video manipulation tools. Hence in the second contribution we propose an effective way of integrating few user-assisted masks to improve detection rates. The third contribution of this paper is an application for reflection detec- tion. Here we explore better feature point tracking for the regions detected as reflection. Tracks usually die quickly in such regions due to temporal color inconsistencies. In this paper we show that the lifespan of such tracks can be extended through layer separa- tion. Results show reduction in missed detections and in computa- tional load over Ahmed et al. Results also show the generation of more reliable tracks despite strong layer mixing