Changing a parameter that isn't on the chain every sample

I am not sure what your goal is. You want a prior that’s fixed but random? The way I would think about it would be to have fixed parameters (if the approximation is not too bad). So you don’t have a fixed A with a special pdf but a distribution with fixed parameters. Say A_mean = 10 and A_std = 1. Then A_noisy = pm.Normal("A_noisy", A_mean, A_std). The latter would change obviously during inference but A itself would be fixed as those are just numbers. Then the prior for A_noisy2 = pm.Normal("A_noisy", A_mean, A_std) in another context is exactly the same as A_noisy1 was. It’s a different way of thinking about it but I feel it achieves the same result, no? If you don’t like to force a Normal prior you can use something else.