Finding people with an audiovisual robot head

Master thesis

Recently, the PERCEPTION team has started to investigate methods and algorithms for scene understanding based on the fusion of visual and auditory data. More precisely, we use the POPEYE robot head that has four degrees of freedom and which is equiped with two cameras and two microphones. We already developed stereo algorithms using the active binocular system, geometric calibration of both the auditory and visual system, as well as localization and detection of audiovisual events based on a new multimodal clustering method.

In this work we propose to investigate new methods for finding the number of people that are present in a room using the POPEYE robot. The method should be able to combine together sound source localization, motion detection and stereo reconstruction to build a dynamic model of the human environment of the robot: how many people are in the room? Where are they located? are they speaking or not?, etc.

The most promising line of investigation seems to be the use of statistical learning techniques, such as mixture models but the student will have full freedom to propose other machine learning methods.

Eligibility: Good knowledge in image and signal processing. Very good knowledge in computer vision and machine learning, as well as very good programming skills.

Start date: 1 November 2009

Contact person: Radu Patrice HORAUD

Deadline: 15 October 2009