Stein Variational Gradient Decsent: RHKS

Stein Variational Gradient Descent: RHKS

Reproducing Hilbert Kernel Space (RHKS)

Before we start SVGD, we should overview some basic concepts in mathamatics. The RHKS will be the first to be introduced here.

Hilbert Space

Definition (Inner Product) Let be a vector space over . A function is said to be an inner product on if

• and if and only if

Definition (Hilbert Space) A Hilbert Space is an inner product space that is complete and separable with respect to the norm defined by the inner product.

Definition (Reproducing Kernel) Let be a Hilbert function space over . A reproducing kernel of is a function which satisfies the following two properties:

• for any .
• (Reproducing property) For any and any .

Theorem Let be a Hilbert function space over . The following are equivalent:

• has a reproducing kernel.
• For any , the function defined by is continuous.

Remark The space of function forms the dual space of .

Definition Let be a Hilbert function space over , we say is a reproducing kernel Hilbert space over if it satisfies the conditions of the previous theorem.

Theorem (Riesz Representation Theorem) Let be a Hilbert space and be a linear functional on (The space of is dual space of ). Then is continuous if and only if there exists such that for any .

Theorem Let . Then is a kernel if and only if is a reproducing kernel of some RKHS over .

Lemma If is a kernel, then it has a feature space and a feature map such that the map (the range of is the set of functions from to ) given by

is injective.

Theorem If is a RKHS, then it has a unique reproducing kernel.

Theorem Let be a kernel on , and let and be an associated feature space and feature map, respectively, such that the mapping is injective, whose existence is guarenteed by Lemma. Then,

• is the unique RKHS with as its reproducing kernel.
• The set is dense in .
• For with and ,

Definition A kernel is said to be integrally strictly positive definition, if for any function that satisfies ,

References

[1] Fox, R., Pakman, A., and Tishby, N. Taming the noise in reinforcement learning via soft updates. In Conf. on Uncertainty in Artificial Intelligence, 2016.

[2] Schulman, J., Abbeel, P., and Chen, X. Equivalence be-tween policy gradients and soft Q-learning.arXiv preprintarXiv:1704.06440, 2017a.

[3] Liu, Q. and Wang, D. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Advances In Neural Information Processing Systems, pp. 2370–2378, 2016