Automated analysis of argumentation in natural language texts

The DISPUTool can be used to compare the arguments of American presidential candidates during televised debates. Here, you can see Richard M. Nixon and John F. Kennedy in 1960. 

This area of research involving language, computer science and machine learning is known as Natural Language Processing (NLP). Argumentation is used across a wide range of publications, including political speeches, news articles, social media posts, medical reports, and clinical trial summaries. These publications have an impact on society, sometimes even on democracy, security, and public health. This is why it is important to study them using NLP techniques.

Mathematical and statistical models are embedded into computer programs, which are then trained using large amounts of data. The goal is to enable these systems to perform tasks such as identifying arguments in a speech or detecting fake news on social media, with a high success rate and without the need for human input. Serena Villata, Director of Research at the CNRS and Head of the MARIANNE project team, insists on the quality of data used: “For the DISPUTool application, that analyses the debates from American presidential campaigns from 1960 to 2016, we used official transcripts. But we had to manually annotate them before using them to train our models. It took us nine months!” DISPUTool is a tool that was designed to facilitate the work of historians and social sciences researchers, by analysing the argumentative content of political speeches.

Automatically detect and analyse argumentations
 

The research projects conducted by the MARIANNE team are focused on three main objectives. The first is to develop computer models capable of automatically identifying all types of arguments in a text, and then classifying and characterising them. This involves detecting specific textual elements and determining whether they represent a fact, an unsupported assertion, etc.

The second objective is to produce an automatic evaluation of the quality of these arguments. Computer processing tools should be able to detect fallacious arguments in a political debate, for example. These are statements that are not necessarily incorrect, but are worded in such a way that deliberately misleads the audience. These tools could also be used to detect hate speech disguised as abusive argumentation in a debate or on social media.

But detecting and condemning such behaviour is not enough. A significant number of sociological studies demonstrate that simply labelling an argument as hateful or fallacious does not change the opinion of the person receiving it. This leads to the third objective, which is to automatically generate counter-arguments, to address online hate speech and hate crime, for example, to clarify fallacious reasoning or expose false information. All of this with reliable, well-founded references.

The MARIANNE joint project team does not aim to develop a specific system, but rather to help boost research in this field, to go as far as possible towards detecting and characterising increasingly complex arguments. Public or private entities could then use these developments on their own platforms, as demonstrated by schools in Trento, Italy, to prevent and address cyberbullying.

Ethics is a key concern 

MARIANNE is a team whose work spans multiple disciplines, including sociology, linguistics and law. This multidisciplinary approach is both highly valuable and important to the team members. “We are eager to explore ethical issues in the digital world”, explains Serena Villata who served for five years on the French National Pilot Committee for Digital Ethics. She currently leads the ANR-funded ATTENTION project, which aims to counter disinformation using AI solutions that are compliant with the ethical and sociological challenges related to disinformation.
 

MARIANNE also has close links with Inria’s “Defence” mission, particularly in the study of disinformation and influence campaigns. The project entitled “Controversy and influence in the Ukraine war: a study of argumentation and counter-argumentation through artificial intelligence”, funded by the ANR, examines the various controversies in English and French that are taking place with regard to the conflict between Ukraine and Russia, in order to understand their impact on the armed conflict on the ground. It focuses on designing and implementing an artificial intelligence algorithm for the automatic analysis of argumentation in these controversies. MARIANNE is also involved in the European ORBIS project, which explores digital democracy models to encourage citizen participation and engagement in democratic processes.