TransAlgo: assessing the accountability and transparency of algorithmic systems
As we are about to book a plane ticket or a hotel room, we can notice a variability in prices just as we proceed to payment. Is this yield managementor is it due to discrimination based on unknown criteria (legal or not)? The aim of the TransAlgo project is to understand, and shed light on, this type of practice. How can methods that make it possible to verify if a decision is taken based on criteria that respect the values and rules of our societies be developed? Nozha Boujemaa, who has been tasked with this major work, responds.
How did the TransAlgo project come about?
In 2016, following the French Law for a Digital Republic, Axelle Lemaire - Secretary of State for Digital Affairs at the time - ordered a report on regulation methods for content processing algorithms from the French General Council for the Economy (CGE). One of the recommendations of this report was the implementation of a collaborative scientific platform aimed at, on the one hand, furthering the development of software tools and algorithm test methods and, on the other hand, promoting their use. We are working on setting up a platform called TransAlgo for the development of the transparency and accountability of algorithmic systems, due to the duality of the data and algorithms.
Inria was given the role of TransAlgo operator, with the scientific contributions of numerous academic players united around TransAlgo's challenges, in particular the CNRS (French National Centre for Scientific Research). In addition to its scientific expertise, Inria will provide support to software development. This platform will be a first in Europe.
Why be concerned with this subject?
A simple question: are the automatic recommendations for the consumption of goods and services (for example Netflix for audiovisual content or Amazon for everyday products) loyal to the consumer or to the service provider? Recommendation engines are becoming increasingly controlling and their transparency now represents a major economic issue, for example for producers of cultural content. Is consent regarding the use of personal data really respected? Subsequently, a recent study by Inria and the CNIL (French Data Protection Authority) caught out a well-known economic player. What was involved was its mobile application, which was overstepping users' consent by communicating their GPS position, despite their refusal. To be specific, the management were not aware of this and had to request an internal inquiry in order to understand the origins of the problem. The lack of program loyalty is not necessarily intentional! Another example of unfair behaviour is the volatile pricing that you can see when the cost of your plane ticket increases along with your visits to an e-commerce website. The aim is not to curb innovation or new business models, but to support innovation via the informed education of the consumer, be they an individual (B2C) or a company (B2B), and through the traceability of automated decision-making. Transparency is an asset for the 'empowerment' of the consumer, but also a factor in economic competitiveness. In the case where the consumer of the service is a professional (B2B), we can allude to a situation of unfair competition - or not.
There are also sorting mechanisms in search engines, proposed content recommendation and selection mechanisms that, at the present time, do not appear in a transparent manner... All of this can have impacts that most people still underestimate with regard to the granting of bank loans, insurance, recruitment situations, etc.
Real challenges, therefore, concerning information, neutrality, loyalty, fairness, non-discrimination, tackling unfair competition, respect for consent and privacy, etc. Nonetheless, one thing that it is very important to understand is that the TransAlgo scientific platform will in no way be responsible for the regulatory control of the algorithms or the use of the data. It will propose studies, tools and services to all of the actors concerned.
Inria was given the role of TransAlgo operator and will be responsible for playing the role of catalyst in the scientific dynamic.
What are the scientific issues of TransAlgo?
The transparency of algorithmic systems is a real challenge for academic research. This calls for several subject-related skills, and many subjects identified have not yet been sufficiently explored by academic research - hence the importance of doubling the research effort. Two approaches will be developed by Transparent: the auditability of the algorithms and the development of new generations of 'transparent-through-building' algorithms that facilitate the measurement of their transparency, their explanation and the traceability of their reasoning. We will also endeavour to develop so-called 'accountable-through-building' algorithms if they respect the laws and if they comply with certain rules and values of our societies.
An algorithm is transparent if its 'accountability' can be easily verified, for example if it opens up its code, if it makes explicit both the origin of the data it has used and the data it produces, if it explains its results, or if it publishes traces of its calculations. It should be mentioned that we will also consider situations where the code is not open since there is no obligation to divulge it.
How are you going to proceed?
In order to be able to set to work on this, it is first necessary to define what we call transparent, neutral, loyal or fair software - notions that are rather legal in nature. This work involves the verification of the compliance between its specifications and its behaviour, in other words the difference between what it is supposed to do and what it actually does. It will also shed light on its compliance with ethical and legal regulations. The methods and technical tools for the transparency of the algorithmic systems are a complex and multi-faceted topic. The properties that we wish to verify, for example non-discrimination or loyalty, involve a significant proportion of subjectivity, which depends on the uses and contexts. This makes their specification difficult. There are numerous scientific challenges, and very little research work exists on this subject.
TransAlgo has a substantial educational role with regard to the general public in order to explain the concepts used, some of which are objectifiable and others not. The TransAlgo platform will provide the scientific community - and beyond - with an area of resources and participatory exchanges. It will also contain white papers, reports and articles in addition to data sets and controlled test protocols. The platform will also be an area for sharing good practices on a national and international level, and a training centre with online courses. The aim is to increase collective awareness of the issues surrounding data and information management and processing algorithms, and to acquire empowering algorithmic tools for citizens, public authorities and professionals.
TransAlgo will also be a facilitation platform for the scientific community, devoted to the challenges posed by issues concerning the transparency and responsibilities of the algorithms. These questions require interdisciplinary expertise and bring together several academic players, in addition to Inria and the CNRS, such as Sciences Po, the graduate school IMT, Grenoble Alpes University, Paris-Sud University, Versailles-Saint-Quentin University, Pierre and Marie Curie University and the graduate school ENS.
We have created five working groups on the following themes:
- Information classification engines and recommendation systems
- Learning: robustness to the bias of data and algorithms, reproducibility, explanation and intelligibility
- Data protection and data use control
- Metrology of communication networks
- Influence, disinformation, impersonation (photos, voice, conversational agent), nudging, fact-checking
How can you be certain of having a foothold in the real world?
We are committed to uses that stem from the lives of citizens and professionals. Associations also have a role to play in the inventory and objectification of the current situation regarding certain practices of the platforms or services by way of contributory measures (citizens and professionals).
In order to gain information on real uses, we liaise with the French Directorate General for Competition Policy, Consumer Affairs and Fraud Control (DGCCRF), the French Broadcasting Regulator (CSA), the French Data Protection Authority (CNIL) and FING (Next Generation Internet Foundation), in addition to CERNA (Allistene's French advisory commission for ethics in ICT research). We also intend to work using feedback on requirements expressed by industry and consumer associations.