Limit or prevent unrealistic output of neural network

Questions
4

  1. If you try printing the logp from the model for all the nodes and there is no inf or nan (see eg Getting ‘Bad initial energy: inf’ when trying to sample simple model), but when you sample using the default trace = pm.sample(1000) it throws an error before the first sample, it is quite likely that the jitter in the default initialization jitter+adapt_diag makes some of the input invalids. Currently you can either set the init='adapt_diag' or init=None. We are in the process of making it more robust.

  2. Yes, you can pass an array to mu and sd, eg:

mu0 = np.random.randn(n_hidden1)
sd0 = np.random.rand(n_hidden1)*2.
weights_in_1 = pm.Normal('w_in_1', mu = mu0, sd = sd0,     
                           shape=(X.shape[1], n_hidden1),
                           testval=init_1)
1 Like