Equipe de recherche PERCEPTION
Audio-Visual Machine Perception
Auditory and visual perception play a complementary role in human interaction. Perception enables people to communicate based on verbal (speech and language) and non-verbal (facial expressions, visual gaze, head movements, hand and body gesturing) communication. These communication modalities have a large degree of overlap, in particular in social contexts. Moreover, the modalities disambiguate each other whenever one of the modalities is weak, ambiguous, or corrupted by various perturbations. Human-computer interaction (HCI) has attempted to address these issues, e.g., using smart & portable devices. In HCI the user is in the loop for decision taking: images and sounds are recorded purposively in order to optimize their quality with respect to the task at hand.
However, the robustness of HCI based on speech recognition degrades significantly as the microphones are located a few meters away from the user. Similarly, face detection and recognition work well under limited lighting conditions and if the cameras are properly oriented towards a person. Altogether, the HCI paradigm cannot be easily extended to less constrained interaction scenarios which involve several users and whenever is important to consider the social context.
The PERCEPTION team investigates the fundamental role played by audio and visual perception in human-robot interaction (HRI). The main difference between HCI and HRI is that, while the former is user-controlled, the latter is robot-controlled, namely it is implemented with intelligent robots that take decisions and act autonomously. The mid term objective of PERCEPTION is to develop computational models, methods, and applications for enabling non-verbal and verbal interactions between people, analyze their intentions and their dialogue, extract information and synthesize appropriate behaviors, e.g., the robot waves to a person, turns its head towards the dominant speaker, nods, gesticulates, asks questions, gives advices, waits for instructions, etc. The following topics are thoroughly addressed by the team members: audio-visual sound-source separation and localization in natural environments, for example to detect and track moving speakers, inference of temporal models of verbal and non-verbal activities (diarisation), continuous recognition of particular gestures and words, context recognition, and multimodal dialogue.
En savoir plus
Retrouvez sur le site web RAweb
- le rapport d'activité complet de l'équipe PERCEPTION (en anglais)
- le rapport d'activité de toutes nos équipes de recherche (en anglais)
- Rapport d'activité complet 2018
- Rapport d'activité complet 2017
- Rapport d'activité complet 2016
- Rapport d'activité complet 2015
- Rapport d'activité complet 2014
- Rapport d'activité complet 2013
- Rapport d'activité complet 2012
- Rapport d'activité complet 2011
- Rapport d'activité complet 2010
- Rapport d'activité complet 2009
- Rapport d'activité complet 2008
- Rapport d'activité complet 2007
- Rapport d'activité complet 2006