Sites Inria

Version française

Technology - Sound - Images

Jean-Michel Prima - 24/11/2010

Automatic evaluation of perceived quality

 © J-M. Prima © J-M. Prima

How can you determine, in real time, whether the video or sound experienced by users is good or bad quality without asking them? In Rennes, research conducted by Inria makes it possible to automate the measurement of this perception. This innovation has an immediate application in the telecoms sector, where the start-up Perceptiva Lab will market services incorporating this technology.

Hollow, distant, choppy or downright inaudible sound. Faded, jerky, frozen or overly pixellated image. The digital era has not done away with all of the sputtering and crackling that disrupt communications. For an operator, therefore, being able to take real-time measurements of how its customers perceive the broadcast quality at any place in the network is clearly of interest. It is not a question of gauging a loss of the signal but instead of quantifying the quality perceived by the user at the point of arrival: ADSL box, PDA, telephone, etc.
Yes, but how? A research team led by Gerardo Rubino, Inria senior research scientist, has recently developed a promising solution. "In principle, perceptual quality is highly subjective", explains the scientist. Each individual feels things in their own way. It is therefore more difficult to calibrate these feelings than to carry out a traditional objective measurement, for example calculating the simple difference between a send time and a receipt time. "
Difficult, but not impossible. "Subjective quality is something we've been able to measure for a long time ."  The trick: panels.  Small groups of people asked about the quality of what they hear or see. "They must indicate whether it is inaudible, difficult to hear, acceptable... For sound, for example, the rating uses a scale from 1 to 5. One might think that perception differs greatly from one individual to another. However, by looking at the group overall, you quickly realise that there is actually some uniformity. Of course, some stand out, but the vast majority ultimately share a similar perception. The group is consistent ."  The proof: "if we test the same sequences with a second panel, the results prove to be substantially the same. Therefore, a truth emerges. Panels make it possible to establish a standard. "

PSQA technology

For Gerardo Rubino's team, the innovation here will involve combining these uniform perceptions with the system status as it is measured at the same time. "We take the whole context, anything measurable and relevant: bandwidth, number of packets lost, etc. Through computing, this learning phase makes it possible to make an objective system state coincide with the perception as shown by the panels. From this point, when we observe a state, we can estimate the quality. An overlap correlates the two levels of information. It is this mapping that opens the door to automation " and to applications in telecoms. In real time, an operator can determine the perceived quality of its network on a large scale. "We no longer measure what 50 people perceive, but let's say... 50,000. "  The technology developed is called PSQA (Pseudo-Subjective Quality Assessment). Who would be interested? Telecommunications operators, content distributors... a considerable market. That said, "we obviously do not offer a universal tool. Perception differs if we watch a video on a PDA or an iPad. It changes if the conversation takes place through a mobile phone or on Skype. It can also depend on the type of coding used. Therefore, for each context, in advance, the system must carry out new learning" with a return to the panels. The main difference with tests using panels is that once this initial phase is completed, the system works automatically, and the measurement of quality is performed almost instantaneously.

Researchers are beginning a collaboration with Perceptiva Lab, a brand new company created by Ricardo Pastrana, with the assistance of Orange Labs. This start-up based in Rennes will offer, among other things, a quality evaluation service to telecoms operators. "Up until now, its know-how focused primarily on the signal. We at Inria also contribute the perceived quality based on the system state, the "network" side . This significant additional dimension is coupled with signal analysis. The two methods share many common points: they lend themselves to the study of network situations, they operate in real time... They are highly complementary. Obviously, we can do a lot of things together. This interesting development in our work also opens the way to even more ambitious things. Because, in addition to information about observed quality, our method also provides information about the system's state: bandwidth, packet loss, delays, etc. And therefore, what we have becomes a true diagnostic tool. This involves not only indicating that there is a quality problem for users but also helping to identify its causes.  A tool for which there is clearly a specific market."

Keywords: DIONYSOS Évaluation automatique Qualité perçue Son Perceptiva lab Inria Rennes - Bretagne Atlantique Images Vidéo