I would go with option 2 - you can add a penalty to the model logp using pm.Potential. It should be a vector the same size as the observed. It is important to get it right if you are using minibatch - you can try setting a theano.shared or minibatch penalty to make sure it is updated each batch the same way as the observed changed.
1 Like