Improving social interactions in the virtual sphere
« As you know, when you speak, you emit vibrations”, says Anatole Lécuyer, head of the project team Hybrid at the University of Rennes Inria Centre. “These vibrations are already used to improve speech recognition for people with hearing difficulties. We thought these vibrations could be of use in the context of the EU programme GuestXR, the aim being to improve social interactions in the virtual sphere. This is something that had never been done before »
Virtual reality headsets with haptic feedback
The team at Inria Rennes achieved a “first”: they developed a virtual reality experiment designed to study the impact of vibrations perceived by an individual when holding a conversation with avatars.
“We gave a group of people virtual reality headsets that would put them into the same immersive space as avatars”, explains Justine Saint-Aubert, a postdoctoral researcher with the project team Hybrid. “Each of them held a vibrator which was able to convert the vocal flow of the avatars into vibrations in the palm of their hand.” Specialising in the use of haptic perception for virtual reality, the young researcher based this on a small vibrator produced by the start-up Actronika, the frequency and amplitude of which will vary depending on the sound signal.
Those sending vibrations to participants 15% more convincing
The first exercise performed using this device involved listening to two identical chatbots, with the same synthetic voice and saying the same insignificant things, ensuring uniform conditions for the experiment. The speech flow of the first avatar didn’t send any vibrotactile feedback to the participants’ hands, while the second sent vibrations that varied depending on the sound of the voice.
Twenty testers then had to fill in questionnaires aimed at evaluating their trust in the avatars as well as their conviction, leadership and social presence (the feeling that they are actually there in the room with you).
We were really surprised at the results”, says Anatole Lécuyer. “We weren't expecting such unequivocal answers. The participants felt 15% more trust in the avatar whose speech was accompanied by vibrations, and felt their leadership was 23% stronger.
Director of Research, Inria
Boosting self-confidence through vibrations
And that’s not all. 20 other participants were asked to take part in a variant on the first exercise. This involved reading a neutral text to two chatbots in turn, receiving vibrotactile feedback when speaking with the first character but not when speaking to the second. Once again, the participants were asked to fill in a questionnaire that was almost identical to the one from the previous exercise.
“We were surprised once again as the results were even clearer. When they felt vibrations while reading the text, participants felt they were 28% more convincing, as though the vibrations were stimulating their self-confidence.” These highly promising results, which were covered in a paper, pave the way for further research and new uses, which the researchers will be presenting for the first time at the IEEE VR conference in Shanghai in late March 2023.
Encouraging people to be kinder to each other in the metaverse
When Justine Saint-Aubert came up with the idea for this “speech-to-touch” device, the team’s aim was to encourage people who are normally reluctant to speak to contribute at meetings (whether virtual or physical, business or personal), boosting their confidence and enhancing their ability to persuade people. They would do this by synchronising vibrations with people’s contributions. “We are keen to ensure our research isn't used inappropriately, whether it’s to further an already authoritarian line or to stigmatise certain points of view”, explains Anatole Lécuyer.
With this in mind, technology will need to be developed with the capacity to activate or deactivate the vibrating devices on request. The aim is to further expand on the team's results, by testing how sensitive individuals are to these vibrations or by measuring the impact of different tones. “The impact of vibration on persuasion that we have demonstrated opens up a huge field of research to explore. This is just the start, but the possibilities are endless”, says Anatole Lécuyer.
GuestXR : encouraging positive behaviour in the metaverse
The research carried out by the Hybrid project team is part of a vast study financed by the European Commission, GuestXR, the aim of which is to tackle cyberbullying and other forms of aggression, in addition to encouraging positive social behaviour in the metaverse. The overarching objective is to develop an immersive online social environment using virtual reality. This will include the creation of a machine learning bot called “The Guest”, which will draw on artificial intelligence in order to analyse the behaviour of participants and individuals, as well as how they behave with each other, the goal being to settle disputes and stimulate interaction.
“We relish contributing to this type of EU project, which involves working closely with other international laboratories”, says Anatole Lécuyer. “As part of GuestXR, we got in contact with IDCH, a team from Reichman University in Israel, to see if we could combine their work on assisting people with hearing difficulties with our work on haptic perception. Had we not done so, we might not have gone down the path of combining vibration with speech. In any case, this led to a fruitful collaboration and a highly promising joint paper.”
GuestXR, a programme coordinated by the EURECAT Technology Centre and the University of Barcelona, brings together four universities, two companies and two research centres: Eurecat (Spain), g.tec (Austria), IDCH (Israel), Inria (France), the University of Barcelona (Spain), Maastricht University (the Netherlands), Reichman University (Israel) and Virtual Body Works (Spain).
Find out more
- Réalité virtuelle : des vibrations qui dopent la confiance en soi - Sciences et Avenir, 23/02/2023
- Quand les objets virtuels deviennent palpables par Anatole Lécuyer et Justine Saint-Aubert, Blog Binaire du Monde, 20/1/2023