iVAE-GAN: Identifiable VAE-GAN Models for Latent Representation Learning

Derosche, Kristoffer Calundan ; Dideriksen, Bjørn Uttrup

4. term

Signal Processing and Computing, Master

2021

2021-06-10

Remarkable progress has been made within nonlinear Independent Component Analysis (ICA) and identifiable deep latent variable models. Formally, the latest identifiability theory enables us to recover the true latent variables up to a linear transformation by leveraging unsupervised deep learning. This is of significant importance for unsupervised learning in general as the true latent variables are of principal interest for meaningful representations. These theoretical results stand in stark contrast to the mostly heuristic approaches used for representation learning which do not provide analytical relations to the true latent variables. We extend the family of identifiable models by proposing an identifiable GAN model using variational inference we name iVAE-GAN. With iVAE-GAN we show the first principal approach to a theoretically meaningful latent space by means of adversarial training. We implement the novel iVAE-GAN architecture and prove its identifiability, which is confirmed by experiments. The GAN objective is believed to be an important addition to identifiable models as it is one of the most powerful deep generative models. We hope such work can inspire other constructions of meaningful latent spaces not based solely on heuristic approaches.

Download
View record in AAU Student Projects