Pymc3.sample() not starting from the specified start value?

Questions
1

I would like to set the start value explicitly when calling pm.sample() using start=, but it seems like the tuning does not use the input. For example, for this toy model:

with pm.Model() as model:
    lnx = pm.Uniform('lnx', 0., 5.)
    infdata = pm.sample(tune=100, draws=0, 
                        start={'lnx': 1.},
                        init='adapt_diag',
                        return_inferencedata=True,
                        discard_tuned_samples=False, 
                        chains=1)

I would expect

infdata.warmup_posterior.lnx.values[0, 0] == 1.

but it instead appears to be a randomly generated number.

pymc3 v3.11.1

2

Interesting… per the docs Inference — PyMC3 3.11.2 documentation the start point might get overridden by the init parameter.

Reading through the sample() function it looks like it should respect and use the start point.

However I do see mutually redundant code at lines pymc3/sampling.py at d7172c0a1a76301031d1b3b411d00643c416a0c4 · pymc-devs/pymc3 · GitHub and pymc3/sampling.py at d7172c0a1a76301031d1b3b411d00643c416a0c4 · pymc-devs/pymc3 · GitHub, which could cause weird behaviour.

I suggest opening a bug report

3

Thanks for the reply! Yea, my understanding is that adapt_diag should not override the start. I’ll make an issue - thanks!

4

Reposting here for visibility of “code doing the right thing” (on github here):

with pm.Model() as model:
    lnx = pm.Uniform('lnx', 0., 5.)
    infdata = pm.sample(tune=100, draws=0, 
                        start={'lnx_interval__': lnx.transformation.forward(1.).eval()},
                        init='adapt_diag',
                        step=pm.NUTS(step_scale=100),
                        return_inferencedata=True,
                        discard_tuned_samples=False, 
                        chains=1)

Note that you have to override the transformed variable, transform your variable, and that the initial point is not recorded (unless it is rejected by the Metropolis correction), so you have to mess with it to guarantee a rejection.

As I said in the issue – probably not a bug, but worth an apology to those who want to use it! Happy to help organize suggestions for improving the API into issues.

1 Like
5

Thanks for the detail @colcarroll - I learned something :slight_smile:

What do you make of the duplicated lines I found?

6

Could definitely use a code comment, but the first one is if there is a step size dictionary provided, it is turned into a list of initial points. That’s news to me that step is eventually a list of dicts, but I guess you could supply something like start=[{'x': 1., 'y': 2}, {'x': 4., 'y': -3.}] if you wanted.

In the second spot it is just after if start is None:, which is the default case, and will initialize start = {}. I think you could get rid of that second if isinstance(start, dict) (since the only way start could be None is if it was initialized on line 520), but it is probably there to be cautious.

7

Aha - I see! Thanks for clarifying that :slight_smile: I guess it could use a comment…