Real-TIME Segmentation foR Advanced Disparity Estimation in immersive Videoconference Applications

Ingo Feldmann, Serap Askar, Nicole Brandenburg, Peter Kauff, Oliver Schreer

Heinrich-Hertz-Institut, Berlin

Einsteinufer 37

10587 Berlin

Germany

{feldmann, askar, brandbg, kauff, schreer}@hhi.de http://bs.hhi.de/

ABSTRACT

We present an advanced disparity estimation algorithm developed for real-time immersive teleconferencing. In order to improve the disparity maps in the case of occlusions a segmentation-driven disparity post processing is applied. We introduce a novel approach for a segmentation algorithm, which uses the advantages of HSV colour space without the need of an explicit colour space transformation. This makes segmentation very fast while still keeping the robustness of HSV colour space especially to illumination changes and shadows. Further on, we present an algorithm for tracking the hands of a conferee. Finally, we introduce an algorithm framework, which combines segmentation, tracking of hands and disparity estimation to create robust disparity map in real-time. We show the improvement of results for image-based rendering.

Keywords: disparity analysis, segmentation, colour space, recursive block matching, real-time