So I’ve (locally) added an implementation of Noisy Natural Gradient in pm.variational.updates which I’ve termed adanoise(): def adanoise(loss_or_grads=None, params=None, datasize=None, learning_rate=0.002, beta1=0.9, beta2=0.999, epsilon=1e-8, kl_lambda=1., gammax=0.1, gammah=0.1): …

Ah you’re right. This was my fault. I was focused on the body of the algorithm, but the first line is if loss_or_grads is None and params is None: return partial(adam, **_get_call_kwargs(locals())) instead of if loss_or_grads is None and params is None: return partial(adanoise, **…

Custom update is automatically replaced by ADAM update?

Questions

ferrine July 3, 2019, 6:38am 4

From initial glance I can’t find where does Adam come from. Is it possible to see sources?

Topic		Replies	Views
How momentum work (ADAM) in the pymc3 framework? Questions doc	0	670	July 1, 2019
Default value used for obj_n_mc in ADVI Questions	1	737	May 16, 2018
Error using init='advi+adapt_diag_grad' update() missing 1 required positional argument Questions	3	1033	December 10, 2019
ADVI giving NaN values to Elbo Questions	5	896	June 17, 2017
ADVI start with initialization Questions	7	2090	September 23, 2017

Custom update is automatically replaced by ADAM update?

Related topics