NoCNN projet: Linking intraoperative images to preoperative MRI

When two researchers from the Camin team, François Bonnetblanc and François Bailly, pooled their expertise—one in cortical stimulation and electrophysiology, the other in computer vision and human movement analysis—the exploratory project noCNN was born.
The goal: to design a tool capable of assisting neurosurgeons during brain tumor removal surgery.

“It all started with a discussion with a neurosurgeon in Paris,” summarizes François Bonnetblanc.

“As soon as you operate on the brain, it deforms significantly,” explains François Bonnetblanc. The idea is to use real images taken during the procedure. “The surgeon can film what is happening on the brain, take photos and videos, and transfer them to the preoperative images to know exactly where they are,” he explains.

The team is thus seeking to compensate for the deformations of the brain that occur during the operation, in order to accurately reproduce the position of the movements, tools, and areas operated on.

“We would like to be able to use digital tools to automatically transfer everything the surgeon does to the preoperative MRI, without changing their surgical routine,” adds François Bailly.

The digital tools developed enable a more accurate correspondence between intraoperative reality and standard preoperative imaging used to plan the surgical procedure, as well as a reduction in the loss of information from the surgical activity itself.

In certain procedures, a deviation of just one centimeter is enough to cause irreversible damage. By aligning images of the procedure with the initial MRI, researchers hope to offer surgeons a more stable and reliable view, enabling them to perform surgical procedures with the utmost precision.

This work, designed to support the procedure, could also be used to better share and teach the complexity of these procedures.

« If you’re not the medical intern observing the surgery, you don’t necessarily understand what’s happening because you don’t have these intraoperative images, you don’t have this connection between the brain before the operation and what’s happening during it,», explains François Bonnetblanc.

By placing these images in context, researchers also hope to preserve a record of this practical knowledge so that it can be passed on and enriched through ongoing operations.

After one year, the team is assessing the scope of the scientific questions to be explored. Real-time preoperative guidance using vision remains a long-term research goal. The exploratory work aims to lay the groundwork, validate technological building blocks, and pave the way for new research and future theses.

Step 1: Surfaces
Recalibration of what is happening “on the surface” using 2D images (photos or single camera).
The challenge: matching a 2D view taken during the procedure to a 3D volume from the preoperative MRI.

Step 2: Going deeper
Taking into account the operated cavity and depths using stereoscopic vision: two cameras attached to glasses are used to reconstruct the 3D image, estimate the position of the tools and the geometry of the cavity, and then recalibrate everything on the preoperative MRI.

The work carried out in 2D forms the basis for future 3D building blocks.

Some advances are already finding practical applications.

In preoperative planning, visualization of veins remains difficult using conventional MRI (2D slice reading). Using tools developed as part of the noCNN project, researchers have begun creating a predictive “photo-realistic” view of the cortical surface (veins, sulci, gyri) — as it will appear upon opening. This component, currently under development, could soon be evaluated in a hospital setting.

Ultimately, embedded stereoscopic glasses could be used to continuously monitor surgical movements and project them onto the surgeon’s control image, providing ever more precise guidance.

The full development of an embedded camera will require research work beyond the duration of the exploratory action, the continuation of which is already under consideration.
A new thesis is in preparation, possibly as part of a joint Inria-INSERM program, with the participation of an expert in computer vision applied to brain images.
Scientific exchanges are also underway with several partners, notably INSERM and CNRS/Inria Lyon, to support the project’s future developments.

The noCNN exploratory project brings together five members: Inria researchers François Bonnetblanc and François Bailly, AP-HP neurosurgeon Prof. Emmanuel Mandonnet, PhD student Paul André, and engineer Émilie Ouraou.

Around this core group, the project involves several medical collaborations, notably with Professor Hugues Duffau and Dr. Sam Ng at Montpellier University Hospital. It is essential to gather as much feedback from the field as possible: practices differ from one department to another, and the approach must be able to adapt to this diversity.

Associated resources
The exploratory action is funding a PhD thesis, which began in the fall of 2024, and an 18-month Technological Development Action (ADT). The thesis addresses the theoretical aspects of recalibration and modeling, while the ADT supports the practical development and implementation of tools.