Excellent!
I am a bit nervous because I expected the uncertainty to grow wider with time, but we see it is a band of constant width. Is my intuition off or is this symptom of a problem?
Actually, your result is exactly what you should expect. As you march forward in time, there is an ever-smaller chance that an attempt will succeed the previous maximum. At some point, the maximum virtually never changes. The width of your posterior predictive on the maximum at this point is then entirely driven by the parametric uncertainty of the underlying normal random variable. The maximum, in the long run, settles down to about the 99.9% quantile of the underlying Gaussian if that makes sense. So the uncertainty band you get should have constant width, the lower bound corresponding to the posterior \mu and \sigma that yields the smallest 99.9% quantile compared to all other posterior samples, and similarly for the upper bound. Of course there is some adjustment to be made since you are really instead taking the lower 5% / upper 95% quantile across all of these quantiles.