Variational Autoencoder

  • Objective of VAE: Maximize $P(X) = \int P(X|\overrightarrow{z};\theta)P(\overrightarrow{z})d\overrightarrow{z}$
    1. Any distribution in $d$ dimensions can be generated by taking a set of $d$ variables that are normally distributed and mapping them through a sufficiently complicated function: $P(X|\overrightarrow{z};\theta)=\mathcal{N}(X|f(\overrightarrow{z};\theta),\sigma^2*I)$
    2. Assume $P(\overrightarrow{z})$ is a unit Gaussian and $P(X|\overrightarrow{z})$ is unit Gaussian
    3. Compute $P(X)$ from $z$ that are likely to have produced $X$
  • Final form of objective function: $logP(X)-D_{KL}[Q(\overrightarrow{z}|X)||P(\overrightarrow{z}|X)]\\=E_{\overrightarrow{z}\sim Q}[logP(X|\overrightarrow{z})]-D_{KL}[Q(\overrightarrow{z}|X)||P(\overrightarrow{z})]$

Other Papers Also Worth Consideration

Some Useful Tutorials


In [ ]: