Generative Adversarial Nets [1]

This is a new framework for estimating generative models via an adversarial process.

A generative model that captures the data distribution; a discriminative model that estimates the probability that a sample came from the training data rather than . In the space of arbitrary functions and , a unique solution exists, with recovering the training data distribution and equal to everywhere.

Adversarial Nets

setting

the generator's distribution over data ,
a prior on input noise variables ,
: a map from z to data space; where is differentiable function with parameters ,
represents the probability that came from the data rather than .

We train to maximize the probability of assigning the correct label to both training examples and samples from ; we simultaneously train to minimize .

When training, it alternates between steps of optimizing and one step of optimizing .

Theoretical Results

Global optimality of

Proposition 1: For fixed, the optimal discriminator is

with this, equation can be reformulated as:

Theorem 1. The global minimum of the virtual training criterion is achieved if and only if . At that point, achieves the value . With this, we obtain:

Princepled Methods for Training GANs [2]

This paper makes theoretical steps towards fully understanding the training dynamics of GANs.

References

[1] @article{goodfellow2014generative, title={Generative adversarial nets}, author={Goodfellow, Ian and Pouget-Abadie, Jean and Mirza, Mehdi and Xu, Bing and Warde-Farley, David and Ozair, Sherjil and Courville, Aaron and Bengio, Yoshua}, journal={Advances in neural information processing systems}, volume={27}, year={2014} }

[2] @article{arjovsky2017towards, title={Towards principled methods for training generative adversarial networks}, author={Arjovsky, Martin and Bottou, L{'e}on}, journal={arXiv preprint arXiv:1701.04862}, year={2017} }