Computer vision recognition pioneer Cordelia Schmid honoured by a prestigious prize

From Applied Mathematics to Computer Science: a 25-year career rewarded

This distinguished and rare Körber European Science Prize is awarded to just one European scientist each year. In addition to its honorary aspect, the prize is endowed with €1 million for future research. Cordelia Schmid is the lucky winner this year. ‘It’s great news, obviously!’ says Cordelia, who is a research director with the joint project team, Willow (ENS-PSL, CNRS, Inria). ‘This is a very select prize and a step up from anything I’ve been awarded up to now.’ 

This is indeed not the scientist’s first prize (see insert): In a career spanning 25 years, she has forged a reputation as one of the pioneers of computer vision.

© Marcus Gloger/Körber-Stiftung

She became involved in digital science at an early age.

I really loved mathematics at school’, she recalls. ‘I wanted to stay in this field and I eventually moved from applied maths to computer science.

An inspirational path from robotic vision to object recognition

During her Master’s in Karlsruhe University (Germany), obtained in 1992, her end-of-study project was devoted to robotic vision.

This incited me to pursue my research in object recognition’, she tells us. ‘It didn’t work at all at that time! It was really difficult for a computer to detect a simple cube, which is a basic task for a human. This pushed me to explore the question further.

She then went on to complete a thesis at the Institut National Polytechnique in Grenoble, on the theme ‘Image matching and retrieval based on local greyvalue invariants ‘. Her work was a major contribution to the sector: ‘This was the first study using grey values to enable algorithms to identify objects in images and the results were very positive’, Cordelia Schmid explains.

Following a 1-year post-doc at Oxford University, the researcher joined INRIA in 1997. Since then, she has remained with the Institute, staying true to her favourite research topics.

In the field of computer vision, there is a whole theoretical aspect with the development of algorithms and models, but there is also a practical side with the direct validation of the programmes we create. I like this mixture of research aspects.

Perception, interpretation and robotics, with a multimedia approach

Currently a member of the Willow project-team (ENS-PSL, CNRS, Inria), she focuses specifically on object and action recognition in videos, with a multimedia approach. And in practical terms? She hopes to create programmes capable of using a video description text, or simply its audio, to learn to recognise the objects in the video autonomously. In other words, to advance ‘unsupervised learning’.

'Instead of providing an algorithm with vast amounts of manually-annotated images to help it learn to recognise an object, we teach it to learn autonomously from video text or its audio’, Cordelia Schmid explains.

The researcher recently returned to her initial study field, robotics. Again, the idea is to encourage algorithm learning, but in this case by drawing on the robot’s interactions with its environment. ‘The aim is to use feedback from the robot’s experience as additional learning data to improve object recognition and the robot’s movement’, the researcher tells us.

Exciting potential which interests the Körber Foundation

There are numerous potential applications for all this research. Some are obvious, such as the development of robots to assist elderly people, clean the home or intervene in a hospital setting. There is also the recognition of specific actions (such as searching for a person eating, playing or swimming etc.) from a database of photos or videos.

Other applications are more discreet, such as the automatic production of high-quality audio-description for a video, or inversely, the generation of images or videos from text. Cordelia Schmid does not target other fields, such as medical imagery or climate change research, but her work may also have great potential in these areas.

All this research and Cordelia Schmid’s successful career are now honoured by the Körber European Science Prize… which also provides a boost to future research.

What does the future hold? More transparent algorithms, multi-task robots and more

‘I’m going to use this prize money to pursue other studies, on the ‘black box’ aspect of algorithms, for example’, says the scientist. At present, algorithms integrate data and deliver a result, but what goes on in between these two steps remains a mystery for the user. Cordelia Schmid would like to add a form of memory to algorithms, which would allow us to collect the data used during the execution of a task.

© Marcus Gloger/Körber-Stiftung

By using both the algorithm and its memory, we could obtain a result and the data which enabled that result’, the research director explains. ‘It’s a way to see the essential factors in the algorithm’s decision-making process, and thus gain clearer insight into how it operates...which in turn will help to improve the system.

The researcher is also interested in the link between image recognition and language recognition algorithms: ‘There are similar issues, so the solutions could be applied to both types of algorithms.’ She would then like to work on robots capable of integrating voice commands, but which move around using vision. While this Körber European Science Prize has rewarded an already long and rich career, there are undoubtedly other prizes in store for Cordelia Schmid’s research.

Find out more

• Körber Foundation press release: Cordelia Schmid (2023): Making AI More Intelligent – Smart Image Recognition for Autonomous Robots.
• Discover the mini-site created by Körber-Stiftung.
• France Is AI, ‘Automatic Understanding of the Visual World’ with Cordelia Schmid, France Digitale (20/11/2020).
• Czech-French #AI Workshop, with Cordelia Schmid: ‘Large-scale Learning from Multimodal Videos’, CIIRC ČVUT (13/9/2022).
• Cordelia Schmid: INRIA – Académie des sciences Grand Prize, INRIA (21/11/2018).
• La reconnaissance visuelle distinguée par le Prix Longuet-Higgins [Visual recognition wins the Longuet-Higgins Prize] (in French), ENS-PSL (2016).