Temporal Matching of Non-Rigid Shapes

PhD thesis

Supervision: Edmond Boyer

Introduction

The Perception team is involved in several research projects related to multiple camera environments, i.e. environments where digital cameras located around a common space can acquire videos of this space, in real time. The proposed position is aimed at solving some of the scientific problems raised by such environments, in particular with 3D model acquisitions over time sequences [1].

Problem statement

This proposal is concerned with applications where videos of the same non-rigid scene obtained from several viewpoints are available. Such videos contains redundant spatio-temporal information on the observed scene. To produce a scene description, usual approaches fuse, or integrate, the information only over the space domain, e.g. standard 3D reconstructions, wrongly assuming that the video information is time independent. In this work, we want to take a different direction and to consider the temporal dimension as part of the process when extracting scene descriptions. Applications of spatio-temporal representations are then numerous and include, among other: scene modeling, which can be improved by temporal consistency constraints; scene animation which can be achieved from videos without the need of prior models such as articulated bodies; medical applications where the analysis of shape motion, of humans for instance, is of great interest but requires temporal correspondences between shapes.

Method

Several spatio-temporal representations have been proposed depending on the information taken into account. For instance Vedula et al. [2] consider optical flows in 2D and fuse them into scene flows in 3D. Correspondances between volumetric reconstruction (i.e. voxels) are then estimated based on this information. Other approaches have been proposed which consider surface elements and correspondences based on photometric and geometric information [3,4,5]. In the first part of the thesis, a thorough study of the different representations and their strengths and weaknesses will be achieved. The second part of the thesis will be devoted to the proposition of a representation and its implementation. As a first application, 3D textured surfaces obtained with the Grimage platform will be considered. Their matching over time sequences will enable visualization and animation applications to be developed.

Expected Results

Expected results are first theoretical: how to efficiently take time into account and which representation should be used for that purpose. They are also practical, solutions will be implemented in the context of the GRAVIR acquisition platform GRIMAGE. Also the perception team is involved in the ARC GEOREP which gather research teams from the graphics, vision and computational geometry community on research themes related to mesh operations. Fruitful collaborations are therefore expected from this ARC.

References

[1] The GrImage Platform: A Mixed Reality Environment for Interactions J. Allard and J.-S. Franco and C. Menier and E. Boyer and B. Raffin, IEEE ICVS, New York (USA), 2006.

[2] Three-Dimensionnal scene flow, S. Vedula, S. Baker, S. Seitz and T. Kanade, IEEE ICCV, Corfu (Greece), 1999.

[3] Spatio-temporal stereo using multi-resolution subdivision surfaces, J. Neumann and Y. Aloimonos International Journal of Computer Vision, 2002.

[4] Sperical matching for temporal correspondence of non-rigid surfaces, J. Starck and A. Hilton, IEEE ICCV, Beijing (China) 2005.

[5] Variational stereovision and 3d scene flow estimation with statistical similarity measures, J.P. Pons, R. Keriven, O. Faugeras and G. Hermosillo, IEEE ICCV, Nice (France), 2003.

Start date: 1 September 2006

Contact person: Edmond BOYER

Deadline: 0000