Interaction
We designed this category to contain the most interactive part of the data set. Human-human as well as human-robot interaction scenarios are inside the category. Each scenario consists of a natural scene in which several human beings interact with each other and with the robot. In some cases one of the actors and/or the robot act as a passive observer. This category contains six different scenarios.
Download
In the following you should find the preview video as well as the download folder for each one of the characters. In the download folder you may find:
- The sequence of the concatenation of the left and right images "sceneXX.mov".
- The left, right, front and rear audio recordings in a four-channel file: "sceneXX.wav".
- The synchronization data: frame number and time instant ("sceneXX.sync").
- Annotation: some of the sequences are annotated. The annotation consists on the image and 3D position of the speakers as well as their speaking state. The annotated sequences are:
Asking for directions
In this scenario an actor asks the robot for directions to the toilets. The robot recognizes the question, performs gender identification and gives the actor the right directions to the appropriate toilets. Six different trials (four male and two female) were performed.
Scene 1
Scene 2
Scene 3
Scene 4
Scene 5
Scene 6
Chatting
We designed this scenario to study the robot as a passive observer in a dialog. The scenario consists of two people coming into the scene and chatting for some undetermined time, before leaving. There is no fixed script and the sequences contain several actions, e.g. hand shaking, cheering, ... Five different trials were recorded.
Scene 1
Scene 2
Scene 3
Scene 4
Scene 5
Cocktail Party Problem
The Cocktail Party Problem has been matter of study for more than fifty years. In this scenario we simulated the cocktail party effect: five actors freely interact with each other, move around, appear/disappear from the camera field of view, occlude each other and speak. There is also background music and outdoors noise. In summary this is one of the most challenging scenarios in terms of audio-visual scene analysis, action recognition, speech recognition and dialog engaging.
Scene 1
Where is Mr. Smith?
The scenario was designed to test skills such as face recognition, speech recognition and continuous dialog. An actor comes into the scene and asks for Mr. Smith. The robot forwards the actor to Mr. Smith’s office. However, he is not there and when he arrives he asks the robot if someone was looking for him. The robot replies according to what happened. Seven trials (five male and two female) were recorded to provide for gender variability.
Scene 1
Scene 2
Scene 3
Scene 4
Scene 5
Scene 6
Scene 7
Introducing people
This scenario involves a robot interacting with 3 people in the scene. There are two versions of this scenario: passive and active. In the passive version the camera is static, while in the active version the camera is moving to look directly at faces of people. Together with the Cocktail Party Problem scenario, they are the only exception where the robot is not static in this database.
In the passive version of the scenario, Actor 1 and Actor 2 interact together with Robot and each other; Actor 3: only interacts with Actor 1 and Actor 2. In the active version, Actor 1 and Actor 2 interact with Robot and each other; Actor 3 enters and leaves room, walking somewhere behind Actor 1 and Actor 2, not looking at robot.Active robot
Scene 1
Scene 2
Passive robot
Scene 1
Scene 2
Scene 3
The Ravel Corpora is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.