Issue on Gaussian Processes approximation

[Yngg-S]

Hi Kaspar,
Thank you so much for sharing your code! I have implemented a tensorflow version and it works well on sin functions. However, when it comes to the functions sampled from gaussian processes, the prediction cannot capture the complexity of the functions. Have you had similar problem?
Thank you!
Yangg-S

[kasparmartens]

Hi!

Sorry for the slow reply. I don't have a general solution, but if your implementation works for sine functions, then for GP draws you could consider larger NNs (e.g. more hidden units) for g and h, and perhaps increasing dim(z) and dim(r), all of these should increase the model capacity. Also, in my experience, replacing sigmoid with ReLU might lead to faster training.

Kaspar