Towards more reproducible research in artificial intelligence for medicine
Caption: areas of the brain "taught" by the algorithm which enable it to determine the probable diagnosis - © ARAMIS
Being able to reproduce the results obtained is a major challenge in biomedical research, and an essential stage in drawing lasting conclusions from them. Olivier Colliot and Stanley Durrleman's ARAMIS team is an Inria team (based at the ICM (Brain and Spine Institute), jointly with the CNRS, Inserm and Sorbonne University) that has developed a set of software tools enabling reproducibility in medical decision-making support system evaluation studies. The results are published in the journal Neuroimage.
The reproducibility of the results is an important issue for the reliability of biomedical research. At present, there is a real difficulty in reproducing the results of the studies, notably linked to the data management and processing methods which are not always sufficiently accessible or standardised. This challenge is present in many fields, in particular the development of artificial intelligence tools for medicine.
The ARAMIS team has addressed this problem in the more specific context of medical decision-making support systems and, in particular, support in diagnosing Alzheimer's disease.
“It is a field in which an enormous amount of work has taken place over the last 10 years or so, with numerous approaches in decision-making support systems. Most of these studies use the same public patient database, ADNI (Alzheimer's Disease Neuroimaging Initiative) and yet it is impossible to compare them with each other”, Olivier Colliot underlines.
Indeed, each study in this field will use a sub-sample of different patients from this database, with criteria that are not always precisely defined. These data then undergo processing that varies from one study to another. The result: it is often difficult to draw conclusions from these studies and their clinical transfer remains poor.
In an attempt to address this problem, the ICM researchers have developed a set of open source software tools in order to very easily standardise and reproduce the evaluation results of these diagnosis support systems. It involves a completely automated workflow, starting from the database, and to which all that is needed is to add the artificial intelligence algorithm developed, for example to distinguish between a patient with Alzheimer's disease from other pathologies, and which will lead to a decision.
At present, this software is intended for the analysis of anatomic MRI and PET (positron emission tomography) data - important tools in the diagnosis of Alzheimer's disease. The researchers wish to continue the development of the software platform in order to extend it to other types of data, but also to other pathologies such as Parkinson's disease.
Olivier Colliot concludes :
We want to move towards a better standardisation of the validation studies of medical decision-making support systems and greater reproducibility, and therefore more reliable results that can be transferred in a more efficient way to the clinic
- Software that is open source and made available by the ICM
- ARAMIS is an Inria team, based at the ICM, jointly with the CNRS, Inserm and Sorbonne University
Sources : Reproducible evaluation of classification methods in Alzheimer's disease: Framework and application to MRI and PET data. Samper-González J, Burgos N, Bottani S, Fontanella S, Lu P, Marcoux A, Routier A, Guillon J, Bacci M, Wen J, Bertrand A, Bertin H, Habert MO, Durrleman S, Evgeniou T, Colliot O; Alzheimer's Disease Neuroimaging Initiative; Australian Imaging Biomarkers and Lifestyle flagship study of ageing. Neuroimage . 2018 Aug 18; DOI:183:504-521.
These articles could interest you:
- European Research Council 2015 Stanley Durrleman receives an ERC Starting Grant
- Research When image processing is used to measure the effects of a drug on the brain
- © Mluz Flores, graphic design & illustration Publication Complex network theory and the brain
- Tractography showing the connections between the different regions of the brain - © Inria / Photo C. Morel Research Using Statistics to Pierce the Mysteries of the Brain
- Interprétation des résultats et du traitement statistique d’images médicales - © Inria / Photo C. Morel Research Observing cerebral degeneration before it shows
- Imagination motrice : contrôler le curseur à l’écran par la pensée - © Inria / Photo C. Morel Brain week How the brain learns to use a brain-computer interface