Reading Assignment 10: Advanced VAE Techniques

In this reading assignment, you will develop a better understanding of the typical problems in VAE training and learn about techniques that aim to remedy them.

(V)AEs

• Start with the nice overview blog post “From Autoencoder to Beta-VAE” by Lilian Weng. (You can skim over the already familiar 1st half.)
• Continue with these blog posts on VAEs, their problems and variants: Part 1 Part 2
• Finally, dive deep with two very recent papers on Preventing posterior collapse with delta-VAEs and Don’t Blame the ELBO! A Linear VAE Perspective on Posterior Collapse.

Further optional readings on VAEs

Read as many papers/posts as you can/want. Many are concerned with general architectural considerations and/or “tricks”, but some are about specific applications (bias towards music). Pick those that interest you.

• Generating Diverse High-Fidelity Images with VQ-VAE-2 - Hi-Res Version (already covered in the blog above)
• Taming VAEs or a similar shorter paper
• InfoVAE
• Wasserstein AEs, along with a follow-up. Bonus question: How are InfoVAE and WAEs related?
• VAEs with spherical latent spaces
• VAEs for learning timbre spaces

Further optional readings on VAE Hybrids

• AE + GAN:
  • Adversarial Autoencoders
  • Adversarially Learned Inference
  • Autoencoding Beyond Pixels
  • Variational Approaches for AE GANs
• AE + Autoregressive
  • MusicVAE
  • PixelVAE
  • Autoregressive Discrete Autoencoders for music
  • NSynth with Wavenet autoencoders
• GAN + Autoregressive
  • ??? Find some :)

Feel free to share additional papers or other resources you come across!