I am not sure if this is more of a question for the Databricks community or the PyMC3 one, but anyway… I’ve just tried to run my first pymc3 model on a Databricks notebook, and the pm.sample command does not show the progress bar… can anyone tell me if there is a way to show it?
That’s interesting, and I’d guess better for the databricks community.
PyMC3 uses the
fastprogress progressbar (link) – if you can’t run their examples in the notebook then it is definitely better for the databricks community!
@evivo did you find a solution to this?
Unfortunately, no. I have checked that the
fastprogress progressbar does not work either and I have posted a message in the databricks forum, no answer so far
since it does not seem that I will get an answer anytime soon, I would like to ask: is there, or would it be possible to have in a future release, an alternative way to visualize the progress?
For instance, by printing a message every n seconds or something like that…
Could you share a small reproducable sample of codes?
I can then run that codes with the latest PyMC version on databrick and get back to you.
Thank you @DanhPhan …
Any code that defines a model and calls pm.sample() would do, for instance the one from this gist
import numpy as np
import matplotlib.pyplot as plt
true_m = 0.5
true_b = -1.3
true_σ = 0.3
x = np.sort(np.random.uniform(0, 5, 50))
y_true = true_b + true_m * x
y = y_true + true_σ * np.random.randn(len(x))
import pymc3 as pm
with pm.Model() as model:
# Define the priors on each parameter.
m = pm.Uniform("m", lower=-5, upper=5)
b = pm.Uniform("b", lower=-5, upper=5)
σ = pm.Uniform("σ", lower=0, upper=10)
# Define the likelihood.
pm.Normal("obs", mu=m * x + b, sd=σ, observed=y)
# This is how you will sample the model.
trace = pm.sample(draws=1000, tune=1000, chains=8, cores=24)
I have run the codes in Databricks, and the results are similar like what you showed. It only show “running…” and the message at the end, without any progress bar.
This is the result using pymc3 version 3.11.4
And this is the result using pymc v4 version 4.0.0b6
However, when I run this notebook in Microsoft Azure notebook, it works. We can see the progress bar as follow:
So, it seems to me that the progress bar is not working on Databricks notebook yet.