Reading Assignment 11: Selected Advanced Topics

This reading is intended to cover advanced architectures that we did not have time for in the respective sessions. Again you cannot be expected to read all these papers, but try to read some introductions, conclusions, and figures.

VAEs

• Neural Discrete Representation Learning aka VQ-VAE
• VQ-VAE 2 for very impressive autoencoder-based results

GANs

• Progressive Growing
• StyleGAN
• StyleGAN 2

This is an excellent series of papers, doing a great job at explaining their choices. They explain architectures as well as all the “little tricks” that really make it work.

Aside from that, these are also interesting works, but we likely won’t have time to cover them:

• CycleGAN for translating between unpaired data domains

Finally, a series on different GAN losses and the gap between theory and practice:

• Wasserstein GAN
• Improved Wasserstein GAN
• Wasserstein GANs Work Because They Fail

Other

• Denoising Diffusion Implicit Models (DDIM) is a technique commonly employed in advanced diffusion architectures as it speeds up sampling significantly.