Monday March 10 , 11h00 - 12h00

Dongheui Lee (University of Tokyo)

3D Recovery from Monocular Observation of Human Motion based on Mimesis Model from Partial Observation

This research aims to capture three-dimensional human motion from an onboard camera system. An approach is proposed to use the database of human motions to recover 3D human motions from a monocular image sequence. Even a stereo camera system suffers from depth insensitivity. Although we assume an onboard camera system composed of a monocular camera, the approach can be extended to an onboard stereo vision system. The technical approach of this paper is characterized by the following three properties. (1) Coordinate transformation of statistical database:　Human motion patterns are usually measured by a motion capture system and represented by a time sequence of joint variables and the 3D position/orientation of the basebody. The hidden Markov model (HMM) is adopted because of a concise representation of spatiotemporal patterns and well established computation. A statistical motion model of the joint variables and the basebody position/orientation is transformed into a statistical motion model in the Cartesian space. A 2D motion pattern from the onboard camera system is referenced to the 2D-transformed statistical motion model in the Cartesian space. By the coordinate transformation, small-scale referencing to the database becomes possible. (2) Concurrent motion recovery and motion recognition:　Motion recovery denotes estimation of the sequence of joint angles and basebody position/orientation from the 2D image sequence. Motion recognition denotes search for the closest HMM (eg. walk, run, jump, and etc.) to the 2D image sequence. For motion recovery, missing information in the 2D image sequence is filled using the searched HMM. The inference cost for motion recognition in the next step is significantly reduced by closing computational loop using recovered motion. (3) Optical flow of feature points:A large computation for image processing of the 2D image sequence would maximally extract information for 3D recognition. This paper proposes a cost-effective computation focusing on optical flow of a rather small number of feature points, being inspired by the MLD experiments. To avoid detailed discussion on feature points selection, we attach artificial markers to the subject as distinctive feature points. Note that the markers are attached at arbitrary points and neither labeled nor tracked like optical markers in motion capturing.