PyMC3 on Databricks: progress bar

Hello everybody…
I am not sure if this is more of a question for the Databricks community or the PyMC3 one, but anyway… I’ve just tried to run my first pymc3 model on a Databricks notebook, and the pm.sample command does not show the progress bar… can anyone tell me if there is a way to show it?

Thank you!

That’s interesting, and I’d guess better for the databricks community.

PyMC3 uses the fastprogress progressbar (link) – if you can’t run their examples in the notebook then it is definitely better for the databricks community!

@evivo did you find a solution to this?

Unfortunately, no. I have checked that the fastprogress progressbar does not work either and I have posted a message in the databricks forum, no answer so far :frowning:

Hello again,

since it does not seem that I will get an answer anytime soon, I would like to ask: is there, or would it be possible to have in a future release, an alternative way to visualize the progress?

For instance, by printing a message every n seconds or something like that…

Thank you


Could you share a small reproducable sample of codes?

I can then run that codes with the latest PyMC version on databrick and get back to you.

Thank you @DanhPhan

Any code that defines a model and calls pm.sample() would do, for instance the one from this gist

import numpy as np
import matplotlib.pyplot as plt


true_m = 0.5
true_b = -1.3
true_σ = 0.3

x = np.sort(np.random.uniform(0, 5, 50))
y_true = true_b + true_m * x
y = y_true + true_σ * np.random.randn(len(x))

import pymc3 as pm

with pm.Model() as model:
    # Define the priors on each parameter.
    m = pm.Uniform("m", lower=-5, upper=5)
    b = pm.Uniform("b", lower=-5, upper=5)
    σ = pm.Uniform("σ", lower=0, upper=10)

    # Define the likelihood.
    pm.Normal("obs", mu=m * x + b, sd=σ, observed=y)

    # This is how you will sample the model.
    trace = pm.sample(draws=1000, tune=1000, chains=8, cores=24)

Thank you!

1 Like

Hi @evivo

I have run the codes in Databricks, and the results are similar like what you showed. It only show “running…” and the message at the end, without any progress bar.

This is the result using pymc3 version 3.11.4

And this is the result using pymc v4 version 4.0.0b6

However, when I run this notebook in Microsoft Azure notebook, it works. We can see the progress bar as follow:

So, it seems to me that the progress bar is not working on Databricks notebook yet.