Combining cameras and inertial sensors for markerless motion capture

Master thesis

The problem of recovering the articulated motion of a person (motion capture) is of great interest from a scientific point of view and has numerous interesting applications in the video and film industries (animation of virtual avatars, inclusion of digital actors for special effects, etc.)

The PERCEPTION group has recently developed such a motion capture system that uses a small number of cameras (4 to 8) The method consists in predicting the pose of an articulated human-body model (with 40 degrees of freedom) and in estimating the joint parameters by comparing edges predicted from this model to image silhouettes. The method performs well but fails in the presence of very fast motions.

In parallel we started to experiment with an inertial sensor which provides rotation information only.

The topic of this project is to investigate methods and to design algorithms able to use inertial information in order to precisely measure motion in joint space and to predict the appearance of the model in camera space.

Based on the developed algorithm and associated software, we will carry out experiments in our multiple-camera laboratory. We will record fast motions (performed by professional athletes) with our multiple-camera system as well as the orientation data provided by the inertial sensors.

Since these inertial sensors are expensive beacuse they are in their initial stage of industrial development, one intersting issue to be studied is the minimun number of such sensors required for robust human motion capture.

Eligibility: Master students (Mastère 2 recherche) with background in computer science, computer animation, and computer vision.

Start date: 1 March 2006

Contact person: Radu Patrice HORAUD

Deadline: 1 November 2006