The problem of segmenting the image and/or the video content into meaningful pieces of information is of crucial importance for further steps such as 3-D shape reconstruction, object and event recognition, etc.

One interesting and promising segmentation paradigm consists in segmenting a video into layers, a static layer (the background) and a dynamic layer (the foreground). This segmentation process can be based on a variety of image cues such as motion, color, depth discontinuities, etc. In the recent past we developped a method for separating moving objects from a static background that is based on a combination of motion detection and color segmentation. The method performs well with both static and rotating cameras.

Another challenging segmentation paradigm is to extract silhouettes. It is well known both in computational vision and in psychophysics that the 2-D silhouettes (as seen in an image or on the retina) of a 3-D object implicitly encodes the geometry of the object. In the case of a moving object, the 3-D geometry can be recovered from silhouettes which are observed simultaneously by several synchronized cameras.

Nevertheless, the automatic extraction of image silhouettes is still a bottleneck thus inhibiting the development of 3-D shape reconstruction. Silhouette extraction is also known as rotoscoping — a widely used technique for motion picture special effects. The output of rotoscoping is a digital matte or a mask which separates the foreground figure from the background. Rotoscoping or matting is currently used for advanced computer animation as well. No effective solution to automatic matting exists today that meets the quality achievable by a human operator.

Within this task we will develop a silhouette extraction method with the following characteristics in mind: robust detection and accurate localisation.

Robust detection of silhouettes will be achieved through statistical modelling of both background and foreground. We will consider the spatial, spectral, and temporal information of pixels to model complex backgrounds. These pieces of information will be used to build a dynamic background model — a model which will continuosly update the knowledge about the background. Classification of pixels into three classes (foreground, background, and silhouette) will be cast into a Bayesian-decision framework.