Learning how to recognize 3D articulated and deformable objects using diffusion kernels
PhD thesis
The problem of object recognition is a central one in computer vision with many potential applications. For the last decade, object recognition has been mainly addressed in the framework of 2D image and video analysis. In general, learning/recognition methods are based on statistical machine learning theory. Images and videos are represented by bag-of-features (a data structure that has been previously used in document classification and data mining), such that each image is described by a vector in a high-dimensional feature space. The most successful learning and recognition methods are based on kernel methods and on support-vector machine (SVM) algorithms, which are generative approaches based on the manual annotation of thousands of examples from a training set. In this PhD we propose to investigate 3D object recognition. 3D objects are generally described by meshes that correspond to uniform and dense samplings of 3D surfaces of objects, i.e., 2D compact manifolds. The latter representation has been thoroughly studied in the recent past and efficient mesh-processing algorithms were developed. There are many practical situations where the input data are gathered using a variety of sensors such as multi-stereo cameras, time-of-flight cameras, or structured-light range cameras. Unlike synthetic data traditionally used in computer graphics and animation, the data gathered with these sensors critically depend on the experimental conditions: They may be corrupted by a significant level of noise and of outliers, they depend on the physical properties of the surfaces being observed, there are missing data due to occlusions and self-occlusions, etc. The temporal coherence of the data is even more problematic in practice: an articulated shape that is observed from different viewpoints or gathered at different time instances may yield completely different samplings and spatial distributions. In this work we propose to investigate the following problems:
How to represent 3D shapes for the tasks of learning, recognition, matching, and tracking.
How to design a weakly-supervised method for learning shape categories from a small training set,
How to efficiently recognize shapes and how to track them over time.
One promising approach is to construct a family of diffusion kernels based on the heat-diffusion equation on Riemannian manifolds. Hence, the problems of leaning and recognition can be addressed within two distinct frameworks: (1) a discriminative approach using kernel-based methods and (2) the discretization of heat-diffusion on manifolds. Therefore, shapes can be represented, analyzed, learnt and recognized in both frameworks, i.e., machine learning and diffusion geometry. In particular we propose to study semi-supervised (or weakly supervised) learning algorithms based on the diffusion kernel. The following tasks will be investigated: Representation of discrete manifold raw-data gathered with sensors (point clouds or meshes) using the eignespace of diffusion-kernel matrices, statistical characterization, construction of local and global shape descriptors, segmentation and registration of shapes, etc.
Eligibility: The candidate should have a Master degree in computer science or applied mathematics with strong background in: computer vision, geometric processing, machine learning, statistics.
Start date: 5 April 2011
Contact person: Radu Patrice HORAUD
Deadline: 5 May 2011
