Learning

XGAN - Learning an interpretable representation of videos generated by a GAN

Date:
Changed on 14/10/2021
Despite remarkable progress in generative adversarial networks (GANs), these networks currently operate as black boxes.

XGAN aims to better understand the black box of GANs for video generation. XGAN proposes strategies to interpret the latent space during the design of interpretable architectures and through the analysis of symmetric functions applied simultaneously on the latent representation and patches of the output generated images.

What does XGAN mean?

The full title of the project is "Interpretable Representation Learning for Video Generative Adversarial Networks".

In this exploratory action, we will focus specifically on answering the following question: "Can we interpret the representations learned by video GANs? ".

Despite the remarkable advances in generative adversary networks (GANs), these networks currently operate as black boxes. XGAN aims to break through the black box of GANs for video generation by proposing strategies for interpreting the latent space in the following areas:

  • the design of interpretable architectures
  • analysis of symmetric functions in input and output of patch-based generation.

How is the project exploratory?

Video generation is a new and difficult problem, and the principal investigator has led one of the few European efforts to address this challenge. Exploring the interpretability and explicability of GANs represents a new and fundamental issue.

The paucity of existing research on video generation is due in part to the vast computational resources required, which we gained access to through GENCI (Large National Computing Facility).

In a departure from previous research, XGAN will explore the interpretability and explicability of GANs, which represent new and fundamental questions, and thus present a high risk.

While the existence of symmetries is given for patches, and they can be analyzed mathematically, it is interesting to know if symmetries beyond the Euclidean transformation will exist. This is a difficult open question, the answer to which will allow the design of more interpretable models.

Do you have academic or industrial partners?

Within XGAN, Edouard Oyallon from CNRS, LIP6, will be a key collaborator, with whom we will analyze the symmetric input and output functions of patch generation. His research focuses on machine learning and on the mathematical foundations of deep learning techniques.

Antitza Dantcheva

Contact

Antitza Dantcheva

Researcher, Stars

2004, route des Lucioles BP93 , 06902 Sophia Antipolis

More on GAN

Article - Tutorial of Razvan V. Marinescu, Medical Vision Group, Massachusetts Institute of Technology